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To  remedy  these  problems,  the  under  secretary  of  defense 
for  acquisition,  technology  and  logistics  issued  a  new  De¬ 
fense  Acquisition  Management  System  instruction  (DoD 
Instruction  5000.02,  Dec.  8,  2008)  and  the  president 
signed  into  law  the  Weapon  Systems  Acquisition  Reform 
Act  of  2009  (WSARA,  May  22,  2009).  Both  actions  seek 
to  ensure  that  acquisition  programs  start  with  realistic  cost 
estimates  and  schedules— based  upon  mature  technolo¬ 
gies  and  designs— in  fulfillment  of  a  defined  and  stable  set 
of  performance  requirements. 

The  purpose  of  this  article  is  to  explain  the  major  tenets  of 
these  new  statutory  and  regulatory  changes  and  to  pro¬ 
pose  new  paradigms  through  which  the  program  manager 
should  think  about  cost,  schedule,  and  performance  when 
starting  a  new  acquisition  program  (see  the  table  on  the 
next  page).  The  table,  Paradigm  Shifts  Based  Upon  DoDI 
5000.02  and  WSARA  of  2009,  depicts  the  new  reviews, 
assessments,  and  requirements  of  the  acquisition  man¬ 
agement  system,  and  is  a  good  reference  as  you  read  this 
article. 

The  WSARA  of  2009  reinforces  much  of  what  was  pub¬ 
lished  in  the  new  DoDI  5000.02,  namely  because  the  Of¬ 
fice  of  the  Secretary  of  Defense  worked  closely  with  con¬ 
gressional  staff  members  to  craft  the  language  in  the  act 
to  ensure  support  to  reforms  already  under  way.  However, 
as  will  be  seen,  the  WSARA  of  2009  goes  further  in  elevat¬ 
ing  the  importance  of  certain  aspects  of  DoDI  5000.02 
reforms. 

Cost  and  Schedule  Considered  in 
Performance  Requirements 

The  WSARA  of  2009  requires  that  Department  of  Defense 
officials  responsible  for  cost  estimates,  budgeting,  and  ac¬ 
quisition  all  weigh  in  on  system  capability  documents  be¬ 
fore  they  are  validated  by  the  Joint  Requirements  Oversight 
Council.  Thus,  the  DoD  director  of  cost  assessment  and 
program  evaluation;  the  under  secretary  of  defense  (comp¬ 
troller);  and  the  under  secretary  of  defense  for  acquisition, 
technology  and  logistics  are  to  comment  on  tradeoffs  be¬ 
tween  cost,  schedule,  and  performance  objectives  as  part 
of  the  requirements  development  process.  This  is  the  first 
major  paradigm  shift  in  how  requirements  for  major  de¬ 
fense  acquisition  programs  are  validated. 

DoD  Instruction  5000.02  reemphasizes  that  “evolution¬ 
ary  acquisition  is  the  preferred  DoD  strategy  for  rapid  ac¬ 
quisition  of  mature  technology  for  the  user."  In  the  new 
instruction,  there  is  just  one  approach  to  evolutionary  ac¬ 
quisition:  incremental  development.  “Spiral  development" 
is  no  longer  used  as  an  evolutionary  acquisition  strategy 
term;  however,  spiral  development  can  still  be  used  as 
an  engineering  term  to  describe  a  software  development 
method.  “An  evolutionary  approach  delivers  capability  in 
increments,  recognizing,  up  front,  the  need  for  future  ca¬ 
pability  improvements.  The  objective  is  to  balance  needs 


and  available  capability  with  resources,  and  to  put  capability 
into  the  hands  of  the  user  quickly." 

To  reduce  requirements  creep,  DoDI  5000.02  requires  that 
“the  Acquisition  Executive  of  each  DoD  Component  shall 
establish  and  chair  a  Configuration  Steering  Board  (CSB) 
...  to  review  all  requirements  changes  and  any  significant 
technical  configuration  changes  for  ACAT I  and  I A  programs 
in  development  that  have  the  potential  to  result  in  cost  and 
schedule  impacts  to  the  program."  Boards  are  empowered  to 
reject  any  changes  and  are  expected  to  only  approve  those 
where  the  change  is  deemed  critical,  funds  are  identified, 
and  schedule  impacts  are  truly  mitigated. 

More  Realistic  Cost  Estimates 

In  the  past,  the  first  cost  estimate  for  an  acquisition  pro¬ 
gram  was  developed  at  program  initiation,  typically  Mile¬ 
stone  B.  This  has  changed  under  the  new  DoDI  5000.02 
and  the  WSARA  of  2009.  Now,  "At  Milestone  A,  the  DoD 
Component  shall  submit  a  cost  estimate  for  the  proposed 
solution(s)  identified  by  the  AoA  [< analysis  of  alternatives]." 
The  emphasis  on  early  costing  of  the  program  is  to  support 
a  Milestone  A  certification  required  by  Congress  (10  USC 
Section  2366a).  In  addition,  the  director  of  cost  assessment 
and  program  evaluation  shall  conduct  independent  cost  es¬ 
timates  and  cost  analyses  for  major  defense  acquisition  pro¬ 
grams  and  major  automated  information  system  programs 
in  advance  of  section  2366a  or  2366b  certifications. 

The  WSARA  of  2009  also  requires  the  disclosure  of  the 
confidence  levels  for  baseline  estimates  for  major  defense 
acquisition  programs.  Justification  must  be  provided  if  the 
cost  estimate  is  calculated  at  a  confidence  level  that  is  less 
than  80  percent.  By  definition,  a  program  estimated  at  the 
80  percent  confidence  level  has  an  80  percent  probability 
of  coming  in  at  that  amount  (or  less)  and  a  corresponding 
20  percent  probability  of  a  cost  overrun.  However,  if  that 
same  program  is  estimated  at  the  50  percent  confidence 
level,  it  has  only  a  50  percent  probability  of  coming  in  at 
that  amount  (or  less)  and  may  experience  cost  growth  over 
time.  That  represents  another  paradigm  shift  in  the  way  the 
military  departments  and  defense  agencies  estimate  the 
cost  of  programs,  as  setting  confidence  levels  to  80  percent 
and  budgeting  to  those  amounts  will  drive  up  acquisition 
budgets,  making  cost  overruns  less  likely  but  also  making 
development  programs  less  affordable. 

Materiel  Development  Decision  Review 

An  initial  materiel  development  decision  (MDD)  review  has 
replaced  the  concept  decision.  In  the  past,  acquisition  pro¬ 
grams  could  enter  the  acquisition  process  at  any  milestone, 
provided  they  met  the  phase-specific  entrance  criteria.  Now, 
an  MDD  review  is  required  first  for  all  potential  acquisition 
programs.  It  is  at  that  mandatory  acquisition  process  entry 
point  that  the  milestone  decision  authority  ensures  that  the 
program  is  based  on  approved  requirements  and  a  rigor¬ 
ous  assessment  of  alternatives.  Then,  according  to  DoDI 
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Paradigm  Shifts  Based  Upon  DoDI  5000.02  and  WSARA  of  2009 


New  Paradigm 

Old  Paradigm 

Cost  and  schedule  must  be 
considered  before  performance 
objectives  are  established. 

Performance  objectives  often 
established  before  cost  and  schedule 
were  considered. 

Costs  estimated  at  80%  confidence 
level  (for  MDAPs). 

With  the  exception  of  high-risk  cost 
elements,  most  costs  estimated  at 

50%  confidence  level. 

Competitive  prototyping  before 
Milestone  B. 

Little  prototyping  because  of  cost. 

Post-preliminary  design  review  and 
critical  design  review  assessments 
for  the  milestone  decision  authority 
make  for  more  robust  systems 
engineering. 

Preliminary  design  review  and  critical 
design  review  were  recommended  as 
"best  practice"  technical  reviews. 

Independent  technological  maturity 
and  integration  risk  assessment 
by  director,  defense  research  and 
engineering. 

Program  manager  assessed 
technology  readiness  level  in 
accordance  with  Defense  Acquisition 
Guidebook. 

Ensure  competition  at  both  prime  and 
subcontract  levels. 

Competition  at  prime  level;  prime 
responsible  for  subcontract 
competition. 

5000.02,  “The  MDA  may  authorize  entry  into  the  acquisi¬ 
tion  management  system  at  any  point  consistent  with  phase- 
specific  entrance  criteria  and  statutory  requirements." 


competitive  prototyping  of  systems  or  critical 
subsystems  before  Milestone  B  approval, 
unless  waived  by  the  MDA.  Yet  even  if  the 
MDA  waives  the  requirement  for  competi¬ 
tive  prototyping,  a  single  prototype  must  still 
be  produced.  In  addition,  the  Government 
Accountability  Office  will  review  all  waivers 
and  submit  their  assessment  of  compliance 
with  this  statute  to  the  Congress. 


Programs  that  have  historically  used  pro¬ 
totyping  in  their  acquisition  strategies  have 
seen  improved  performance  and  increased 
technological  and  design  maturity.  The  effort 
to  produce  a  prototype  also  helps  in  under¬ 
standing  development  and  production  costs 
and  aids  in  the  refinement  of  the  program 
cost  estimate.  However,  even  a  single  pro¬ 
totype,  not  to  mention  multiple  prototype 
contracts,  can  drive  up  development  costs. 

During  the  TD  phase,  statute  and  regula¬ 
tion  also  require  that  major  defense  ac¬ 
quisition  programs  conduct  a  system-level 
preliminary  design  review  (PDR).  Per  DoDI 
5000.02,  "A  successful  PDR  will  inform  requirements 
trades;  will  improve  cost  estimation;  and  identifies  remain¬ 
ing  design,  integration,  and  manufacturing  risks." 


Materiel  Solution  Analysis  Phase 

The  materiel  solution  analysis  (MSA)  phase  has  replaced 
the  concept  refinement  phase.  While  an  MDA  decision  to 
enter  the  new  materiel  solution  phase  doesn't  mean  that  a 
new  acquisition  program  has  been  initiated,  the  new  term 
implies  that  some  type  of  material  solution  is  being  pursued. 

The  AoA  is  the  key  activity  of  the  MSA  phase.  DoDI  5000.02 
calls  for  a  more  robust  AoA  than  in  the  past.  "The  purpose  of 
the  AoA  is  to  assess  the  potential  materiel  solutions,  identify 
key  technology  elements,  and  estimate  life  cycle  costs,  in 
order  to  satisfy  the  capability  needs  documented  in  the  ap¬ 
proved  initial  capabilities  document  (ICD)."  The  AoA  must 
also  assess  appropriate  system  training  and  alternative  ways 
to  improve  energy  efficiency.  Additionally,  resource  esti¬ 
mates  must  use  the  fully  burdened  cost  of  delivered  energy 
in  trade  off  analyses.  As  mandated  by  the  WSARA  of  2009, 
the  DoD  director  of  cost  assessment  and  program  evalua¬ 
tion  develops  the  AoA  study  guidance  for  major  defense 
acquisition  programs. 

Technology  Development  Phase 

The  name  of  the  technology  development  (TD)  phase  was 
not  changed.  However,  both  the  WSARA  of  2009  and  DoDI 
5000.02  require  competitive  prototyping  in  that  phase. 

In  a  significant  paradigm  shift  for  major  defense  acquisi¬ 
tion  programs,  acquisition  strategies  must  now  provide  for 


The  cost-performance  trades  that  result  from  knowledge 
gained  during  competitive  prototyping  can  help  keep  the 
program  affordable  and  within  the  Milestone  A  compo¬ 
nent  cost  estimate.  A  post-PDR  assessment  by  the  MDA 
is  also  required,  and  its  purpose  is  to  establish  the  allo¬ 
cated  baseline  for  the  system  and  to  approve  requirements 
trades. 

The  TD  phase  is  guided  by  the  ICD,  draft  capabilities  de¬ 
velopment  document  (not  stated  in  DoDI  5000.02,  but 
implied),  and  the  technology  development  strategy;  and  is 
supported  by  systems  engineering  planning.  "The  project 
shall  exit  the  TD  Phase  when  a  affordable  program  or  in¬ 
crement  of  militarily  useful  capability  has  been  identified; 
the  technology  and  manufacturing  processes  for  that  pro¬ 
gram  or  increment  have  been  assessed  and  demonstrated 
in  a  relevant  environment;  manufacturing  risks  have  been 
identified;  a  system  or  increment  can  be  developed  for 
production  in  a  short  timeframe  (normally  less  than  5 
years  for  weapon  systems);  or,  when  the  MDA  decides  to 
terminate  the  effort,"  according  to  DoDI  5000.02. 

The  WSARA  of  2009  also  requires  an  independent  as¬ 
sessment  by  the  director  of  defense  research  and  engi¬ 
neering  of  the  technological  maturity  and  integration  risk 
of  the  critical  technologies  of  major  defense  acquisition 
programs.  In  addition,  the  director  of  defense  research 
and  engineering  is  to  develop  knowledge-based  standards 
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against  which  to  measure  the  technological  maturity  and 
integration  risk  of  critical  technologies  at  key  stages  in 
the  acquisition  process.  In  the  past,  the  program  manager 
was  responsible  for  technology  readiness  assessments 
that  were  based  upon  definitions  provided  in  the  Defense 
Acquisition  Guidebook.  While  the  director  of  defense  re¬ 
search  and  engineering  has  yet  to  announce  its  technologi¬ 
cal  maturity  and  integration  risk  standards,  one  can  expect 
them  to  be  different  from  the  Defense  Acquisition  Guidebook 
definitions,  perhaps  requiring  knowledge-based  evidence 
from  testing  in  order  to  meet  the  standards. 


Engineering  and 
Manufacturing 
Development  Phase 

The  engineering  and  man¬ 
ufacturing  development 
(EMD)  phase  has  replaced 
the  old  systems  develop¬ 
ment  and  demonstration 
phase.  The  new  name  for 
the  phase  implies  that  the 
system  (e.g.,  prototype) 
works  and  is  ready  to  be 
engineered  into  a  produc¬ 
ible  design.  It  is  in  this 
phase  that  tools  and  tech¬ 
niques  are  to  be  developed 
and  demonstrated  for  the 
manufacturing  of  the  sys¬ 
tem.  A  key  objective  of  the 
EMD  phase  is  to  establish 
the  product  baseline  for 
all  configuration  items, 
resulting  in  more  empha¬ 
sis  on  systems  engineer¬ 
ing  and  technical  reviews. 


In  the  past,  th 
estimate  for  an  acquisition 
program  was  developed 
at  program  initiation, 
typically  Milestone  B. 
Now,  "At  Milestone  A, 
the  DoD  Component  SRff 
submit  a  cost  estimate  for 
the  proposed  solution(s) 
identified  by  the  AoA." 


The  EMD  phase  is  guided 
by  the  capabilities  develop¬ 
ment  document,  acquisition  strategy,  systems  engineering 
plan,  and  test  and  evaluation  master  plan.  The  acquisi¬ 
tion  strategy  is  prepared  by  the  program  manager  and 
approved  by  the  MDA. 

The  EMD  phase  consists  of  two  efforts,  the  first  of  which 
is  the  integrated  system  design  (ISD)  that  is  intended  to 
define  system  and  system-of-systems  functionality  and 
interfaces,  complete  hardware  and  software  detailed 
design,  and  reduce  system-level  risk.  ISD  includes  es¬ 
tablishment  of  the  product  baseline  for  all  configura¬ 
tion  items.  Completion  of  that  effort  is  evidenced  during 
a  system-level  critical  design  review  (CDR),  conducted 
by  the  government  program  manager  and  the  contrac¬ 
tor.  Following  the  CDR,  a  mandatory  post-CDR  assess¬ 
ment  has  replaced  the  old  design  readiness  review.  Its 
purpose  is  to  tie  the  product  baseline  to  a  decision  by 


the  MDA  to  continue  into  the  second  effort  of  the  EMD 
phase.  Elevating  the  post-CDR  to  the  MDA  level  is  ex¬ 
pected  to  strengthen  the  systems  engineering  effort. 

Systems  capability  and  manufacturing  process  demon¬ 
stration,  the  second  effort  in  the  EMD  phase,  is  intended 
to  demonstrate  the  ability  of  the  system  to  operate  in  a 
useful  way  consistent  with  the  approved  key  performance 
parameters,  and  that  system  production  can  be  supported 
by  demonstrated  manufacturing  processes.  “This  effort 
shall  end  when  the  system  meets  approved  requirements 

and  is  demonstrated  in  its 
intended  environment  using 
the  selected  production-rep¬ 
resentative  article;  manufac¬ 
turing  processes  have  been 
effectively  demonstrated; 
industrial  capabilities  are 
reasonably  available;  and 
the  system  meets  or  exceeds 
exit  criteria  and  Milestone  C 
entrance  requirements,"  ac¬ 
cording  to  DoDI  5000.02. 

As  was  the  practice  under 
the  old  version,  the  new 
DoDI  5000.02  requires 
that  programs  entering  the 
EMD  phase  be  fully  funded 
in  the  future  years  defense 
program.  That  means  before 
entering  the  EMD  phase  at 
Milestone  B,  all  of  the  dol¬ 
lars  and  manpower  needed 
to  carry  out  the  acquisition 
strategy  have  to  be  included 
in  the  budget  and  out-year 
program.  Obviously,  a  pro¬ 
gram  that  is  only  partially 
funded  is  more  likely  to  fail. 

The  WSARA  of  2009  requires  that  the  secretary  of  de¬ 
fense  ensure  competition  or  the  option  of  competition— 
at  both  prime  contract  level  and  the  subcontract  level — 
throughout  the  life  cycle  of  the  program,  as  a  means  to 
improve  contractor  performance.  While  ensuring  compe¬ 
tition  at  the  prime  contract  level  is  not  new,  guidance  on 
government  involvement  in  subcontracting  competition 
has  been  strengthened.  The  law  requires  that  the  govern¬ 
ment  ensure  fair  and  objective  “make-buy"  decisions  by 
prime  contractors  on  major  defense  acquisition  programs. 
Government  surveillance  of  contractor  sourcing  decisions 
and  the  assessment  of  sourcing  fairness  and  objectivity  in 
past  performance  evaluations  are  also  mandated. 

Under  the  new  DoDI  5000.02,  test  activities  are  inte¬ 
grated  into  every  acquisition  development  phase  for  early 
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JOIN  THE  SUCCESS  NETWORK 

The  DAU  Alumni  Association  opens  the  door  to  a  worldwide  network  of  Defense 
Acquisition  University  graduates,  faculty ,  staff  members,  and  defense  industry 
representatives— all  ready  to  share  their  expertise  with  you  and  benefit  from  yours. 

Be  part  of  a  two-way  exchange  of  information  with  other  acquisition  professionals. 

•  Stay  connected  to  DAU  and  link  to  other  professional  organizations. 

■  Keep  up  to  date  on  evolving  defense  acquisition  policies  and  developments 
through  DAUAA  newsletters  and  symposium  papers. 

■  Attend  the  DAUAA  Annual  Acquisition  Community  Conference/  Symposium 
and  earn  Continuous  Learning  Points  (CLPs)  toward  DoD  continuing  education 
requirements. 

Membership  is  open  to  all  DAU  graduates,  faculty,  staff,  and  defense  industry 
members.  Itrs  easy  to  join,  right  from  the  DAUAA  Web  site  at  www.dauaa.org. 

For  more  information ,  call  703-960-6802  or  800-755-8805, 

or  e-mail  dauaa2(at)aol.com. 


m  I  ■ 


identification  and  correction  of  technical  and  operational 
deficiencies.  The  new  instruction  also  requires  that  the 
deputy  under  secretary  of  defense  for  acquisition  and 
technology  conduct  an  independent  assessment  of  op¬ 
erational  test  readiness  for  all  ACAT  ID  and  special  inter¬ 
est  programs. 

For  programs  on  the  Office  of  the  Secretary  of  Defense 
Test  and  Evaluation  Oversight  List,  the  director  of  opera¬ 
tional  test  and  evaluation,  in  coordination  with  the  program 
manager,  determines  the  number  of  production-represen¬ 


tative  or  production  articles  for  live  fire  test  and  evaluation 
and  initial  operational  test  and  evaluation.  There  can  be 
significant  costs  and  schedule  impacts  associated  with 
those  test  articles  and  tests. 

A  Better  Acquisition  Program 

The  new  DoDI  5000.02  and  the  WSARA  of  2009  man¬ 
date  changes  to  the  acquisition  management  system  to 
fix  mismatches  between  requirements,  cost  estimates, 
and  budgets.  The  new  MDD  review— required  for  all  pro¬ 
grams— added  emphasis  on  the  AoA,  and  a  component 
cost  estimate  at  Milestone  A  should  help  to  harmonize 
actions  in  the  requirements  budgeting  and  acquisition 
management  systems.  Knowledge  gained  from  mandated 
competitive  prototyping  should  also  help  detect  immature 
technologies  and  inject  more  realism  into  early  cost  esti¬ 
mates.  If  implemented,  cost-saving  trades  identified  dur¬ 
ing  prototyping  can  help  keep  program  costs  within  initial 
cost  estimates.  Likewise,  configuration  steering  boards 
can  help  put  a  stop  to  changing  or  excessive  requirements 
growth  and  help  contain  cost.  Finally,  full  funding  upfront 
for  required  test  articles,  statutory  tests  and  evaluations, 
and  formal  technical  reviews  will  give  new  development 
programs  a  better  chance  at  succeeding. 


The  author  welcomes  comments  and  questions  and  can  be 
contacted  at  william.fast@dau.mil. 
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